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ABSTRACT 

^ The J^SltOTrSation Exchange Procedures (lEP) developed 
by the National Center for/fti.gher. Education Hanageaent systeas 
(NCflESS) are a set of standard definitions and procedures for , 
coilect£hg infora&tiqn about disciplines and student degree programs^ 
outcbaes of instructional programs, a»nd general institutional 
cSiaritcteristics. A fundamental purpose o^ lEP is to assist 
postsecondary education institutions ip establishing. a basic set of 
dat^ useful for the iihprovement of education planning and management, 
This^urpose- is puriSued throug^ IFP in two ifays:^(1) by helping th^^ 
institution to crea^te its onrC internally' useful data set^ and '(2) by 
-'^fostering he creation and exchange of comparable data sets among 
siail^^r institiitionsv The -lEP basic d'ata set noy consists of 
iTiforaation of each of the following types: (1)/ institutional. / 
characteristics; (2) studl^it enrolji^ents and characteristics; (3)/ 
personnel resources; (4) other resources; (5) 'institutional fin^nices; 
J6) costs l?y instructional discipli^ie a:nd student prpgraa; and/*(7) 
student outcomes. This document is ^directed toward the definition aad 
discussion of lEP student outcomes inforaatioii and prdcedur^s for 
obtaining them 

set necessary ^ , . 

Qverviewv of the outcomes of student programs; and allows Ahe 
institution in the context of ISP to Wd<^ttake examiiiation of the 
activities^ costs, and results of tho^e programs. (i'uthor/KE) 

/-^^ 



It also presents th^ procedures ^or creating the data 
to support further developments, provides a /descriptive 
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♦ • PREFACE . \ . 

/ ' 

the Information Exchange Procedures (lEP) tfeveloped ty the National Center 
for Higher Education Management Systems (NCHEMS) are a^set of standard 
definitions and procedlares for collecting iJvJtonnaition about disciplines and> 
student degree programs , -outcomes of instructional prograpis, and, general- p 
institutional characteristics. These definitions and pro«dures have been ^ 
developed to facilitate exchange of J^formation among instit^ions of post- 
secondary ^ucation, piroviding institutions with sorne' assurance that data they 
e)^'charige are useful for purposes of comparison. 

Tfie procedures and definitions recommended in this manual .and other manua^ls -about 
lEP are "targets" for institutions^ While^t is anticipated thSt most institutions 
will be able to follow these recommendations', some institutions may not have 
available the required ^information and others may be'abl6 to provide i.t only:*t 
an unreasonable cost. Others may find t>iat only after several years of using; 
lEP can data be developed that accurately reflect the characteristics o^ the 
institutions- For some institutions the definitions and procedures may be too 
complex; others may find them too simpJistic. ^ny effort at exchange of data 
among institutions involv^es some? compromise in an attempt tq accommodate the 
wide variationiamong. institutions. Consequently, while t:omparable Information 
for exchange is the goal of lEP, NCHEMS cannot guaXntee absolute comparability 
of data as the final result of lEP. But, hopefully, implementation of lEP will' 
represent reasonable progress tow^d that goal.^ 't ^ . 



. . 7 
The .Information Exchange Procedures initially were developed by the NCHEMS 
staff with guisdance from a talsk force and ^steering, committee, .composed of 
institut^mial^and <sta agency representati/^s. The cost procedures were 
tested ^during 1972-73 by a group of about 60 cortmunity colleges, private 
colleges, .and state colleges and universities. The full set of exchange 
procedures mi tested during 1973-74 in about the same number and kinds of 

institutions. Insights gained fromlthese pilot tests served to refine, the 

I 

full rarKja^f procedures and definitions: 




The.Iitformation Exchange. Procedures described in this manual, and m others 
listed below, h^ve been reviewed and approved by the lEP advisory groups and 
the NCHEMS Board of Directors, and are being* released at this time for wide- 
scale-implementation. While lEP is intended to be a final product, widescale\ 
implementation may point to ^additional Information needs of institutions that , . 
can-be addressed in updates" to this manual if necessary. However, NCHEMS feel$ 
that lEP is sufficiently refined at ^his time that it can be released to* the 
full NCHEMS general distribution mailing list* and to other institutions 'interested 
in implementing lEP. . . ' 



This 'publication is one of four implementation manuals for lEP: * . 
j| ^ I nfoy]{jfati on 'Exchange Profcedures Activity Structure , Technical Report 

No. 63, specifies. the structure to be used in collecting institutionc^t ^ 

" * • • • 

data', including detailed examples of an account, crossoveh. This /. . 

* * : * * * * 

'structure is based on the NCHEMS Prograrn^ Classification Structtrnfe , ' 
T^hnical Report No. 27, as modified in late 1974. / 
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Information Exchange Procedures Data Formats anc! Definitions , Technical 
Report No. '64, contains the formats recommended for collection and 
display of the lEP data set'. This document also includes a complete 
glossary of lEP terminology arid 'definitions. « 

Information 'Exchange Procedures Cost Study Procedures , Technical Report 
No. 65, contains the procedures for implementing the cost study portion 
of lEP with specific references to<the NCHEMS Costing' apd Data Management 
System. The cos ting, procedures specified in this document are'baseclon 
those developed in the Cost Finding Principles project, v , v . 

Information Exchange Procedures Outcomes ^tij i v Procedures , Technical 
Report No. 66, contains the procedures for implementing the outconies 
study portion of lEP. Th^^ outcomes measures and prdceduresAspecified 
in this do/ument are' based on those developed in the Outcomes of Post- 
« secondary Education project at NCHEMS. 

' • *. * \ J* * * 

These documents replace the following documents: ' 

^ - ' 

Information -Exchange Procedures* Manual (Field Review EdHionhv>rA 

\ * ' -f 

Synopsis , Technical Report No. 46 . , • ' 

Igformatioji Exchange Procedrrres Manual (Field Review Edition) , Technical 
Report No. 47 . ^ 
' Information Exchange Procedures CosF^tudy Imp;lementation GCiide (Pre- 

~ i ; '. ~ I ^31 7 

liminary Edi tion) , Techiiioil Report No. 52 ^ \' 
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The' software system designed to support 'the Information Exchange Procedures 'is> 

\ • 4. 

'documented separately. Readers concerned with the NCHEMS Costing and Data 

Management System should refer to the following documents: 

An Introduction to the NCHEMS Costing and Data l^anagement 5^vstfim ^||^ 

Technical Report No. 55 ^ ^ - 

NCHEMS Costing and Data Management System—Sample Reports . Technical 
. Report Np. 56 . ' ^ • ... 

Account Crossover Module Reference M&mial , Technical Report No. '57 
Faculty Activity Module Reference Manual , Technical Report^No: 58 
PersonneT Data Module Reference Manual, Technical Report No- 59 

1 ; : >\ " . . 

Student Data Module Reference Manual , Technical Report' No. 60 ' 
Student Outcomes Module Reference Manual , Technical* Report No. 61 . 
Data Management Module ReferetKe Manual , Technical Report No. 62 

■ • ■ i-' ' ~ 

other NCHEMS documents jthat will be useful in conjunction w.ith this manual are:- 
An Introduction to the Identification and Uses of Higher Education 

Outcome Information , Technical Report' No. 40 4 
Higher Education Outcome Measures Identiffcation Study: A 
^^es griptive Summary . 
Outcome MeSTsures and Procedures Manual: Volume I (Field Review 
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' INTRODUCtlON 



A fundamental purpose of the NCHEMS Information Exchange Procedures (lEP) is to 

assist postsecondary education institutions in establi^ing a basic set of data 

useful for the improvement of educational banning and management. This purpose 

is pursued through iW^in two ways: ^(1) by helping the institution to create 



its own internally useful data set and 
change of comparable data sets among si 



(2) by fostering the creation and ex- 
lilar institutions— the simple premise 



being that institutions as well as individuals may learrr from each other. 
One of the initial objectives of lEP was to establish a methodology for developing 



and exc^ianging information on educational costs ^ and tfujs a substa^'ntial portion of 

\ , ! 
lEP documentation has addressed the issues and procedures associated with cost 

' 9 \ i 

\ ' 

calculation. .An additional objective»was^ the development of a broader data 
set, one composed of mbre than cost information alone. This development was 
subject to two general guidelines: . " I 

- The information included in lEP should help users to understand 
better the activities and programs of the'institution. ; 

- The information included in lEP should encourage institutions to 
perform analyses that will help them in the evaluation and 
improvement of their programs. ^ 

The resulting lEP basic data set now consists of information of each of the 
following types: \ 

\ 



^ ♦ ^ ' c 

• Institutional characteristics 

** * ' * ' - 

- Student eftrollments and cliarJicteri sties 0 ^ -^^ 

• Personnel Resources * ' * . 

^ ' ' i * 

• • Other resources * • ^ * , . , 

• Institutional finances . , 
-nCosts by instructionlil <liscipline and. student program 

- Student Outcomes 

• ' ■ - \ ■ ' - ■ 

This manual is <lirected toward the definition and discussion of lEP stud nt out- 
comes infontiati^n and procedures for obtaining it. " / * 

Objectives of the lEP Outcomes Study 

The lEP Outcomes Study has three broad objectives: 

1. To .advaftce-^he state of the art in institutional planning and managei|tent 
by providing" a^ more fully developed set of standard student outcomes 
measures than has previously been available to a large number of diverse 
institutionsrr^ . ' ' 



"^'tunons m rne aes*..riDtion or stud.^ 
by providing definitions of outcome measures "and standard proc^ures 



2. To assist institutions iri the des*..ription of student program^outcomes 



tandard 

for obtaining the necessary information. ' \^ * 
3. To present the j'nformation in such a way that' institutions are encouraged 
to untjierlake analyses and comparisons of instructional program results 
and examine relationships to other pISgram descriptors and activities. 
The ultimate objective, of course, is the actual use^of outcomes information— 
in both a descriptive and a comparative mode— as an essential' element*^ in program 
planning and management. That objective requires the further development, by 



■I" '." / in Sel 



NCHEMS and others, of analytical approaches to outcomes -data and specific ^ 
suggested uses of otitcomes information in institertlonal problem solving. 
(Some of the^ possibilities for this kind of development are discussed briefly 



I ... Sejct- tfp III-;) This document presents the procedures for creating /the data 



set* necessary to support .those further developments- More importantly for 

/ ' 1 • ' » 

»/ the present, it ecovides a descriptive overview of the outcomes of student 

» ■ ^' . * t. 

programs and allows tlre^insti^ution, in the context of IE|, to undertake 

exgiftjinatiorr pf ^the activities-^ costs, and results of those programs. 



\ 



^ ' c OUTCOMES MEASURES AND PROCEDURES 

/ • . *^ \ The lEP Outcomes Measures 



The lEP Outcomes Study focuses on the results of student programs i and in this 
respect it differs from^some (but not all) of the other components of lEP. 

The institution's instructional resources, for example, usually are associated 

^ . - / 

with a particular discipline or, perhaps a multidiscfplinary department. The, 
outcome measurers in this document, ort^ the other hand, are associated^rwith the 
curHculum or course of study* followed by tHe student—that is,' the student 
program. ' ' ' . • 



Most of the measures included here were derived frrom or influenced, by the Out- 
comes of Postsecondary Education Project at NCHEMS. TFris project is an ongoing 
and substantial ef^rt of the Center aimed art* the^ identification and measurement 
of the outcomes'of postsecondar/ institutions and programs, and at promulgation^ 
of the ijse .of outcomes information in educational planning and management. 
.(The outcomes project activities that contributed iji a major way to this docu- 
ment are dekribed .briefly in Appendix A*) The content 'of this manual thus 
has a firm base in the broader and more intensive work that continues in the 
^Outcomes project. Nonetheless^ one must be cayreful ^not to overlook the limited 
natupe^f the outcomes measurement capability represented by the current 
prbcedures . 



v. 




Firsts it should be noted that the outcomes measures includjed here relate only 

to student growth and deV^^lopment* (That is, there are no measures of research* 

*• 

or public service outcomes^) This is consistent with the remainder of l£P at 

this time, in which majqr (but not exclusive) attention is devoted to the s 
institution's instructional activities and their components. -'In addition, 
these Student growth measures permit the institution to examine student program 
outcomes that are more than simple ipedsures of productivity (such as student 
credit hours)* Thirty such student growth measures were given 'very high- 
priority in the Outcome Measures. Identification Study (see Appendix A), and 
these served as the initial set for possible inclusion in lEP^ 

Feasibility criteria caused further limitation of the lEP outcomes data set* 
One measure on the high priority list, for example, was "Student scores on 
tests that indicate their ability to read, write, speak, and listen." The 
inclusion of this measure in lEP would require first the development of tests 
of conniunication skills or the adoption of one^or more existing tests. It 
would require also that , institutions participating in lEP be willing to accept 
such tests and administer them to their students. This was regarded by the lEP 
staff and advisory bodies as an unacceptable requirement for institutional " if 
participants. ^ j 

After feasibility constraints were applied, there remained a set of outcome 
measures that received high ratings in the Outcome Measures Identification 
Study, and that allowed reasonable ease of collection. These comprise the lEP 
outcome measures: 



\ T. The number of program completer? for the previous year, by type o^ 

^ ^ , • . ^ . • 
certificate or degree awarded and field of study. 
*- ^. . - ^ 

2. The number of students who left the institution in the previous year 

* « * '* 

< prior to program completion, by type df certificate or degree sought 
and by their status at 6xi|:\^ / 

3. The time elapsed between: entering the institution and program completion. 

4. Whether the student transferred credits from another institution. 

The number of terms spent tn completing the program, full -time and ^ 

partrtime. ' 

6. The cumulative grades-point average of program completers. 

7. The number of program ccmpletei^s seeking a full-time job. ^ , 
,8. The number of program completers securing a full-time job and * 

a. the.typV of job. 

b. .the perceived permanence and career potential the job.* 

c. the starting salary or wage. ^ 

d. ' the relation of the job to the' field of study.' 

9-. The Yong-run career expectations of program completers. 

10. The number of program completers applying for admission to another 
degree program and , ' . ' 

a. the type of dearee sought \artd field of study. 

' . ' i 

b. the number of applicants- who, have been admitted. *\ J 

11. The long-run educational intentions of program completers witf] respect to 

' ' ^' 

^ type of degree and field of study. 



6 

ERIC 



1:2; "Program complete;rs;' ,p^fcp,ptfons of the institution's contribution to 
their progress, and the^irftpprtance of :tfta t prog res St i n -si >C 'a rear ^o f : 
potential grm^ • " \ \ - 

a*<e Intel! ertuaV growth. :\ • / 

, . ' ' . tj ' ' ' * 

•b. SoGtal growth ^ • * ' > * 

c. Aesthetic an(i^,<^uVturaV^grow,th ' :;>' ... * 

d. EducafionaT>;grbw|:h ' * ■ 'i\ • 

e. Vocational aad- prpfe^^^^ growth. 

The Student Outcjgmes Questionnaire for Program Completers 



Early j*n the development of outcomes procedures for lEP; it became clear that 
most of the important measures could be obtained readily—and perhaps most 
reliably—fhnough a 'sl:udent -survey. The decision was made therefore to develop 
a standard questionnaire, printed by NCHEMS and made available to lEP partici- 
pating institutions, for administration to program completers. (This questionnaire 
appears on thet^lnside^back cover of this document. A brief summary of ,its; 
development and 'instructions /or obtaining copies are in Appendix Bj 

> 

Two factors were of primary importance in reaching this decision. (1) A single 
standard questiojipavre would enhance comparability of information across 'programs, 
and institutions, whereas even minor variations in wording' or Tn the order of 
items from one institution to another would diminish comparability. (2) A single 
standard questionnaire allows NCHEMS to provide computer software, as part of the 
NCHEMS Costing and Data Management System, that wi^fl^edit questionnaire* responses'' 



"17 



and. perform tfie calcu>ations-necessaryt.tO-^omplete-the lEP data forma'ts. The 



resulting machine-readable record of the questionnaire dat^.also will jFacili- 
tate later analyses. This kind of assHstance for participating institutions 
was a major consideration. 

•5- 

The decision to provide a standard student outcomes questionnaire is not meant 
to discourage institutions from adding items. In fact, it is anticipated-that 
many institutions will want to ask questions unique to ^heir programs or* 
actiyi^^es—questions that cannot be accommodated in a general questionnaire 
of this s^rt. Others may want to explore in greater depth some of the variables 
that are already included. Many of these ideas are under investigation in thje 
tICHEMS Outcomes project, ar^d interested institutions are encouraged to contact 
^the NGHEMS staff working on the Outcome^i; projeci, either to request documentation 
of project. developments or to share tlie institution's experience with additional 
outcome measures. 



^. Implementation of lEP Outcomes Study Procedures 

The procedures for the lEP outcomes study fall naturally into two categories* 
those that rely on institutional records and thos'e^at rely on the student 
outcomes questionnaire for prbgram completers. c* 

ft 

Institutional Record-Based Information is summaf^iz^d in Format F.l, Student put- 
comes InformaJ^ion\* Program Enrollments' and Completions. This format shows the 
first two outcome measures listed earlier: (1) the numbpr of program completers 



/ 8i8 



anii (2) "the number ofWiting students'* (noncompleters) by their status at exit. ' 
'Although it is not an outcamd measure, headcount enroll ment-^is'^ presented also ' 
for purposes of comparison. The instructions and definitions for Format f;.l - 
are given on the facing pW. Although collection of this pprtion'of the 
lEP outcomes data is essentially straightforward, a few points nee* to be stressed. 



The?e are illustrated throilgh the use of sample data in the formats in Appendix 



1. It 'is int^ded,that an 



entry be.majde for tha total irxstitution,^ 
followepi by a separate entry for each studenlrprogram, idjentified by v , 

^ the relevant code from *the tHEGIS .Taxonomy. ; \y^' ^ * ? 

• • • '-^ * 

2. Many institutions will haye a substantial number o^ stude>its enrolled, 

pakicul^arly at the lower-division level, who have not yet declared 
. a st/ident major. Tiiese. students should be listed separately as 
' "uTWeclared." - ^ . • • . • 

3. Each entry should be d^*saggregated by type of degree or certificate. 

' * Fo'r headcount ^enrollments and exiting students (noncompleters)/this # ' 

11 be the degree or certificate toward which the student is (or 



was)- working. For. program completersV. this will be the degrae'or 
c^tific^^^B^tually awarded. \ - / 

4.. Tile institution should use its own' discre,tion in deciding whether it 
j's useful to provide an additional . breakdown vrithi^R the lEP standard 
degree types., (For some purposes, for exampleV the institution may 
wish to separate Associate of Arts degrees from Associate of Science 
'an!^ Associate of Appfied Science degrees.) ^ v " 

5. It! is expected that in many institutions some student programs will 
have very few enrollments. In these cases it may be useful to group 
the data in categories that are of interest— perhaps by college divisions 
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■ or other organizational units.. Note, however, that these aggregated 
entries should .be in addition to- -not in place of—the discrete pro- 
gram entries. Note also^that the aggregations used, if any, should 
be consistent with those in the cost study portions of lEP. 

6. Finally, for exiting students (noncompleters) it may be difficult to 
extract data on the student's status at exit or the student's program 
and/or degree type. The institution is urged, nonetheless, to "secure 
as much of this information as possible, ^even if^only* totals can be given*. 

student Questionnaire Information is summarized in Format F.2, Student Out- 
comes Information: Outcomes ^u^sy<Jnna^re for Program Completers. Agafin, 
instructions and definitions for completing the format are, given on the facing 
page'. The necessary information for Format.f .2 is generated from questionnaire, 
responses 'by the NCHEMS computer software for editing^and reporting ;the 
questionnaire data (see the Student Outcomes Module Reference Manual , Technipal 
Report No. 61). Two aspects of the procedures for producing this questionnaire 

information are discussed here: questionnaire administration and questionnaire 
■ 

data processing. 

I 

A.y Questionnaire Administration : * The following are recommended procedures 
for administering the student outcomes questionnaire. 

1 : Timing 

\ The questionnaire should J)e administered at the^end of each term 
^ in which degrees and certificates are awarded. If the institution 

is limited' to a one-time administration for the year, it should 
i choose the term in which the^«largest number of awards usually is made. 
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Questionnaire adrninistratiort should take place no later than the 
degree or certificate award date for the term and no earl i er 
than 60 days befor^:^the award date. Ideally, responses would be 
obtained in the month before' program completion, \ 

2. ^ -Swyey Population . 

xThe population surveyed should include all program completers^as 
, described on the page facing Format FJ. It is rrecognized that 
some Institutions will consider it desirable to survey a sample 

program completers rather -than all of them; A particular 
sampling procedure is not recommended here, but the student 
program orientation of the lEP outcomes study mandates that the 
• sampling procedure adopted take account of the different numbers 
of completer?* in the various programs a>id degree types. In any^ 
event, if the entire population of completers is not included in, 
the survey, the institution should keep carefuV documentation of 
. the sampling procedure used, Th'is -is important so^that differences 
in questionnaire administration-can be identified and accourjted for 
at a later tfhie if the institution wishes to undertake trend 
analyses of student outcomes. It is important also that such 
differences be identifiable when one institution exchanges outcome 
information with another, 

3, Method of Administration 

The method of administration of the questionnaire must Jbe an 
institutional decision. In general, the possible methods are 
of two types: mailing or direct handout. 




t . ■ . . • 

A mailed questionnaire, even with a follow-up, probably will 
yield, a lower response rate than a handout, perhaps because of 
its- more impersonal nature. In addition, there are added 
expenses in the form of postage costs. If the mailing method 
mjst'be used, the following procedures are recommended, 
(1) If the institution has the capability, a preprinted label con- , 
.taining the student's name and stadent LD, number should be 
attached to the'questionnaire. This wi,ll allow monitoring .of 
responses for follow-up and also ensure that questionnaire 

V 

responses carry the same LD, number as other student files at ^ ' 
the institution. If the label conta.ins the student's local 
address as well, it also may be used as a mailing label in an 
envelope with an address window. (2) Since the questionnaire ^ 
is designed with one blank side, it also can be utilized as a • , ' 
."return mailer." That is, the^stitution can pre-label the 
blank side with the address to which the questionnaire is to 
be returned and add the needed postage, rather than enclosifig 
a return envelope. (3) At least one follow-up of nonrespondents 
should be conducted approximately two weeks after the initial 
mailing. A postcard reminder probably is suitable for this purpose. 
If there is concern about respondents misplacing the initial question 
naire, however, then a second questionnaire and return envelope 
should be sent. - /V " 

Handing out the questionnaire is the preferred method, but the 
circumstances under which the method is applie^will vary con- 
siderably. In some institutions, a classrooin handout and collection 
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will be feasible and will likely yield «a very good rate of 

« 

r>* response • Other institutions wi.ll prefer to irake use of the 
counseling or placement office, the academic departments, the 
degr^-registration. section of the registrar's office, or some 
other central location. Whatever the circumstances^ of question- 
naire distribution, two cautions should be borne in mind. (1) Care 
should be taken, that the uie of a particular distribution site 
does not bias the response* ' If the placement office^, for example, 
is the only location wheni" the questionnaire is available, 
the respondents will consist only of those who are seeking jobs* 
(2) The time of administ/ation of the questionnaire may be more 
important than the response rate in determining the amount of 
^__ija#e«nation^that is^jireful to the J^hstitution* It maty be 

conv|nient, for example,>/^ have|t^e sturfSht complete the question- 



naire at the time of degree 'registration; in ^itio^, such =i pro- 
cedure would be likely to capture a very "high percentage of program 
completers. If, however, degree registration takes place ver^ early 
in the term (say January or February for a award date),, many 
Respondents will be unable to provide information\on their plans 
following program completion. 

Again, depending on the handout methocj employed and on institu- 
tional capabilitj^, it may be convenient to use a preprinted label 
for monitoring questionnaire returns* - 
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Cover Letter 

the use of an institutional cover letter attached to the 
questionnaire is strongly recommended. The cover of the 
questionnaire itself contains a note to the student briefly 
explaining the purpose of the study Because of Its standard 
form, however, there, is nothing that links the questionnaire 
directly to the institution. A cover letter will serve this 
purpose, will establish the importance of the response for the 
particular school, aK^^robably will produce a better response 
rate. The cover letter can serve also j:o introduce any 
additional qiiestions the institution may wish to include . 
in the survey ♦ thus making the whole process seem le&s remote 
to the s1;udent. ^ . 

* V 

Follow-Up 

The need for a follow-up 'Of nonrespondents wilj .depend cnfthe 
method of administration and on the rate of response that is 
important to the Institution. As indicated above, if the 
questionnair^ is mailed, at least one follow-up^-probably in 
pos^t card; form— is recommended. Of course, this procedure can 
be used also if the questionnaire is handed out to^be returned 
later. Other procedures include telephone foUew-up, classroom 
and campus newspaper announcements, contacts through departmental 
advisors and faculty members, and the like. 

- ' 29 ' 
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B. Questionnaire Data Proc e ssing : The following guidelines are intended 
to assist in monitoring iquestionnaire returns and preparing them for 
input to the Stuident Outcomes Module' software. 



1 . Monitoring Responses 

It is recommended that questionnaire monitoring and data 
processing take place in a central institutional location*. 
Returned questionnaires may be collected in several different 
* offices, such as the instructional departments', but a single * 

office should have responsibility for Identifying those wKo have \ 

» 

.'responded. In some cases this can be done by a computer check 
against the file of students registered for program completion. 
In others ^ manual comparison will need to be macle against a 
master list. The monitoring process can be carried/out more"^ 
easily if returned qi^^stionnaires .are stored in order, according 
to the student LD. number. (This ordering also will facilitate 
the quick location of responses that art^rejected by\he editing 
software.) 

% • , If the percentage of nonrespondents is large, the institution will" 

want to know something about who they are; they ma/T^r example, 
constitute homogeneous groupings that result in severe under- 
representation of some student programs. »Thus it is important 
to keep careful records of those who respond and those who do not, 
even in those cases where a follow-up of nonrespondents is impossible. 
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2; Hand Checking \ ■ * . 

Before turning the questionnaires over for Reypunching, it would be 

• good to check some of* the responses by hand. If a large number of 

* oddities occur in the respgnses as a whole (such as blank , sections, 

extraneous writing, and Ojther inattention to instructions), i\^ is like/ly 

that some of them will appear in the process of this visual check* 

Further checking and culling of the .responses should make it possible ^ 
♦ \ • 

to give complete instructions to the keypuncher (concerning what^ to do 
with the "oddities," for exa^mple). This, in turn, may save keypunc;h 
time^and result in an improved data set. 

3. Keypunching 

The student outcomes questionnaire is designed in such a way that 
additional coding is. needed prior to keypunching the information,* pus, 
^ once hand checking is completed, the questionnaire responses are ready 
to be put in machine-readable form for input to the computer software 
module. The outcomes sof^tware will edit the questionnaire data and 
produce the information needed for lEP data format F,2. In addition, 
it creates a raw data file, containing all questionnaire information for 
each student, for later analysis, A full description of the Student 
Outcomes Module software is included in theP Student Outcomes Module 
Reference ^^^nual (Technical Repo^t^ No. 61) and the NCHEMS Costing and 
Data Management System— Sample Reports (Technical Report No, 56), 




*The bracketed number following each item number on the questionnaire indicates 
the keypundi card column, Note that four record cards are needed for the 

' questionnaire information: one for the respondent's name (item 1), two for . 
the address* (item 2), and one for the response data (itenjs 4 through 25). 
Each card contains a 7-digit record identifier (explained in the Student 
Outcomes Module documentation) and a 10-digit (maximum) student identifier • 
"taken from item 3. The questionnaires information then is punched beginning 
in column 18 on each card (see items 1, 2, and 4). . 
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The Student Outc&mes Module software will produce an edit report 

listing all rejected item responses according to the identification 

number of the respondent. (An example of a rejected response, would 

be^ the us^e of an occupational code that does not appear on th& list 

, • on the questionnaire.^ These rejected item responses will be 

considered blanks (tha^ij-is, no^esponse) in the latwr computations 

unless they are -changed. Before proceeding, then, the institutional 

user should examine the edit report to determine .whether there are 

any rejected ftem responses that he or she may want to change. 

(For example, an illegal occupational code could be changed to a 

999~"0ther"— and the response included in the computations. In 

addition, this examination will reveal a situation in which a 

particular respondent provided a largis number of illegal responses, 

in which case the user may want to delete the entire set of answers 

for that respondent. 

4 

> * « 

The procedures and guidelines discussed in the paragraphs above constitute the. 

^ I 

recommended approach to student questionnaire administration and handling for 
the lEP outcomes study. It must be stressed, however, that these procedures 
are heavily dependent on the institution's' resources, its commitment to the 
study, its perceptions of the importance of the information, and the kinds of 
decisions for which that jnformation will be used. Sbme institutions will 'be 
severely limited in the amount of staff time and money they can devote to the 
study. ^Others may be able to spend substantial effort in achieving high 
respbnse rates and in analyzing and using the data. At both extremes, it is 
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felt that the information resulting from -Use of "the student outcomes question 
naire represents a significant increase in the institution's student data 
base and potential Understanding of "its instructional program outcomes. 
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AflALYSES AND U^tS^OF lEP OUTCOMES DATA ' . 

\ 

As it was indicated in the first. section, this manual is not an analysis 
document. Rather it is intended, as part of lEP, to support the creation 
of a data base on which outcomes analysis can be conducted. Much of this 
analysis must originate with the institution since it Is the .institution 
that wiJI be in the best position to identify questions of interest. It 
is hoped 'that the very process of collecting and examining this outcomes 
information— in a way that may not have been done before— win help in 
suggesting analyses. , _ : 

Centering as it does on student^programs rather than departments or disciplines 
£6^ se, the information resulting from these procedures may provide an additional 
planning and management perspective; It can be a vital link in examining' 
what kinds of students choose particular programs (and why) and what' kindstof 
later "success'' measures are associated with 'them. These questions suggest 
the possibility of surveys of entering students (to learn more about their 
demographic characteristics^ their attitudes, and their plans), surveys of 
program noncompleters (to learn why they left the institution and what they 
haW-.done in the time since leaving), and follow-up surveys ofp graduates (to 
learn how their educational and occupational plans are actually carried out). 
Even without these additions, however, this data base permits examination of 
student outcomes similarities and differences among student programs within 
\:he institution and eventually among student programs between institutions. 
At the very least then, the institution will have a comparable set of 
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information about its\student programs. Beyond that it may have some \ \, 

valuable clues about areas needjng attention and areas that may serve as 

program "models." ^ : 

• * » I 

As may be expected, NCHEMS \s not unconcerned with the direction and quality 
of the institutional analysis t\iat is undertaken with lEP outcomes data. In 
the course of the broad implementation effort, NCHEMS staff will work closely 
wi^fv selected institutions in implementing this set of outcomes procedures 
and in analyzing the resulting data— both as a cohesive set of outcomes infor- 
mation and in conjunction with the other kinds of TEP information. To the ' 
extent that these efforts yield exemplary approaches for other schools, and 
to .the extent that participating institutions are willirlg, theig results will 
be shared with other institutions implementing lEP/ 

) 

In addition, considerable IflCHEMS effort is being devoted to analyzing results 

» ' * * 

Of the 1974 pilot test of lEP, on which the current. lEP procedures are 
substantially based. This staff effort, based on actual institutional data, 
will provide added insight into the kinds of questions institutions may wish 
to address in the course of th'eir own analyses. 

Finally, NCHEMS has under way a comprehensive examination of institutional 
"uses of the lEP data set. Two approaches are being developed: (1) examina- 
tion of a single kind of information (for example, the outcome^s data set) across 
many student programs and (2) examination of comprehensive information 



(for example, student,, resource, cost, and outcomes data) for a single student 
program* The. two approaches suggest different— perhaps complementary- 
applications- of lEP data. These applications will be further enhanced by 
a parallel NCHEMS effort devoted to examination of comparative analyses and 
uses of multi-institutional lEP data sets, and the development of an exchange 
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Aiechanism whereby institutions may share their lEP data v^ith others. 

' ' ' ' '/ 

/' 

dearly then, this'document is far from the last word, c/n outcomes ^ev^n in 
the; context of lEP. While much remains to be done, however, implementation 
of these procedures should provide substantial benefit to the institution in 
terms of improved understanding of its student programs and their results. 
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The NCHEHS Outcomes Project 

As stated in the text, most of the measures included in, this manual were 
derived from or influenced by the Outcomes of Postsecondary Education Project 
at NCHEMS. Three major milestones in the Outcomes project were particularly 
important in this context. The first of these was the staff's compilation 
of the Inventory of Higher Education Outcome Variables and Measures (in 
Micek and Wallhaus, 1973). This inventory lists and describes a wide array 
of h\'gher education outcomes and suggests potential measures or proxy 
measures for them; the section comprising student growth and development 
outcome variables is especially relevant to the lEP Outcomes Study. The 
second milestone was the Higher Education Outcome Measures Identification- 
Study (see Micek and Arney, 1975), in which some 300 educational decision 
makers at both the institutional and state level reacted to a list of 112 
outcome measures. In this study the -respondents rated each proposed measure 
In terms of their perceived need for information about that outcome in 
fulfilling their pfofes'sional responsibilities. The results of'these 
ratings were used to establish an initial set 'of high priority measures for 
potential inclusion in lEP. The third important milestone was development 

» 

of th.fe Outcome Measures and Procedures Manual: Volume I (Micek, Service, 
and Lee, 1975). This manual, which will be the subject of a pilot test 
during 1975, presents suggested procedures for obtaining information related 
to the highest priority outcome measures from the Outcome Measures Identifica- 
tion Study. Some of the procedures in that document. were used in the 
preparation of the present lEP Outcomes Study Procedures. 

X 

• / 
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Continuing efforts in the Outcomes project include ^ 

- Development of a conceptual fram^prk for describing and understanding 
the full range of outcomes of postsecondary education* • 

- Development of useful measures of outcfifnes of the institution's 
research and public service activities and programs. ' 

- Exarrnnation of potential uses of existing- outcome measures at the 
state level. ^ 

0 

- Definition and development of outcome measures^ having particular * 
utility for noncollegiate and nontraditional institutions. 
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The Student Outcornes Questionnaire for Program Completers 

Several steps were imp^tant in the development of the Student Outcomes 
\ Questionnaire for ProgrSn Completers. First, ,a sample ^student questionnaire 
and procedures for its administration were included in the 1974 pilot test 
version of lEP. Next, a small number of 1974 pilot test^schools were 
selected' to give particular attention to the use of the student questionnaire; 
^hese >jncluded the Comnunity College of Philadelphia, Nohnandale Community 
College, Seattle Community 'College, West Vifginia Wesleyan College, and 
the Universities of Evansville, Notre Dame, and Rhode Island. Each school 
- used the basic items from the lEP sample student questionnaire. * Some added 
Items of particular Interest to their institution and explored alternative 
wordings of some items. Different methods of a^inistration also were 
examined.* 



The results of the pilot test, along with additional' development by the 
Staff and recommendations from the task force of the NCHEMS Outcomes project, 
substantialjy^affected the present version of the questionnaire. 

- The items concerning immediate occupational ^ind educational plans were 
clarified. ^ 

- Item^ were added concerning long-run occupational and educational 
plans. 
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♦Continuing analysis of the pilot XssJi-^/^n ence is being carried out in 
the NCHEMS Outcomes projett. ■ 
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- The litems dealing with educational, progress were combined^ into broader 
categories of student growth and a question was added concerning the 
importance of -that progress to the respondent* 

- The questionnaire was constructed so that it was unnecessary for the 
institution to compile from its records any information, alout the 

• ■ ' • \ ' ' 

respondent. ' \ 

list of occupations and educational programs was provided to permit 
•precoded responses to mos^t of the items. 

Three demographic items were added: birth date, sex, and civil 
rights category.* 



*The civil rights category headings and definitions used in the questionnaire 
and this manual are adopted from recommendations made to the. Ad Hoc Committee 
on Racial and Ethnic Definitions of the Federal Interagency Committee on ' 
Education (FICE). At the time of the piTe!^(aration of this .manual , the 
recommendations had not y^ been subm1tted\for endorsement by the full fICE 
committee. What is presented here must therefore be regarded as tentative; 
it is,' nonetheless, the most timely and substantive coi^tribution to a 
revision of civil rights categorizations. 

American Indian or Alaska Native : A person having origins in any of the 
original peoples of North Apierica. 

Asian or Pacific Islander : A person having origins in any of the original 
peoples of the Far East, Southeast Asia, or the Pacific Islands. This area 
includes, for ex^ple, China, Japan, Korea, the t^hilippine Islands, and Samoa. 

Black/Negro : A person having origins in any of the original peoples of 
sub-Sahara n Africa. 

Hispanic : A person tff Mexican, Puerto Rican, Cuban, Central or South 
American, or other Spanish culture or origin, regardless of race. 

White, other than Hispanic : A person having origins in any of the original 
peoples of Europe, North Africa* the Middle East, or the Indian subcontinent. 
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/ 

> . : ■ /. 

The ^ re vised questionnaire was informally tested by sta^F members from NCHEMS 

and ''the office of 'the Assistant Vice-chancellor for Academic Affairs at 

/ 

/ 

the University of Colorado. / 

/ 



INSTITUTIONS WISHING TOj OBTAIN COPIES OF THE STUDENT OUTCOMES QUESTIONNAIRE 
FOR PROGRAM COMPLETERS MAY DO SO BY CONTACTING: 



The Publications Unit 

Western Interstate Commission for 

Higher Education 
P. 0. Drawer P 
Boulder, Colorado 80302 
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OUTCOMES DATA FORMATS WITH SAMPLE DATA 
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STUDENT OUTCOMES QUESTIONNAIRE 
EOR PROGRAM COMPLETERS 



TO THE STUDENT 

i 

This cjuestjonnaire is part of a study you4^chool is undertaking in order to 
understand better some of the nrjost imporVont outcorries of postsecondary 
education — its effects on studems. Your responses to the questions con- 
tained here will provide a broad picture of your educational and occupational 
goals and' plans. This in^ornriation in turn will help your institution to provide 
the best possible educational program for its students. 

The questionnaire is brief, but your response to it is very important. Your 
answers will be tabulated along with those of other students. All the infor- 
mation you furnish will be treated confidentially. Thank you* for your help, 
and best wishes. 
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